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Abstract 

We provide a simple example showing that some conditional information inequalities (even in a 
weak form) cannot be derived from unconditional inequalities. 

1 Introduction 

Three conditional information inequalities were considered in 0[4j[8]: 

(1) I{A:B\C) = I(A:B) = I(C:D) < I(C:D\A) + I(C:D\B) 

(2) I(A:B\C) = I{B:D\C) = => I(C:D) < I(C:D\A) + I(C:D\B) + I(A:B) 

(3) /(ArSjc) = H (C\AB) = => 7(C:£>) < J(C:D|A) + J(C:D|B) + 

Here A,B,C,D are random variables (with a finite range), H(U\V) stands for conditional Shannon 
entropy of U given V, and I(U:V\W) is the amount of mutual information in U and V when W is 
known. 

Question. Can these conditional inequalities be derived from their unconditional versions? 

For example, inequality (1) could be a consequence of an unconditional inequality (with "Lagrange 
multipliers") 

I{C:D) < I{C:D\A) + I{C:D\B) + k[I{A:B\C) + I{A:B)} 

if the latter were true for some n > 0. This is not the case, as shown in [H] (for all three inequalities 
(1-3)). In this note we consider a statement that can be considered as a conditional inequality that is 
weaker than both (1) and (3)0 

I{A:B\C) = I{A:B\D) = H(C\AB) = I(C:D\A) = I(C:D\B) = I{A:B) = I(C:D) = 

We give a simple algebraic example that shows that even this weak statement is essentially conditional, 
therefore providing an alternative simple proof of the conditional nature of inequalities (1) and (3). 

Theorem. There is no constant k such that 

I(C:D) < k [I(A:B\C) + I{A:B\D) + H{C\AB) + I(C:D\A) + I(C:D\B) + I(A:B)} (*) 
for all random variables A, B,C, D. 



*Authors are supported by ANR NAFIT grant 008-01 

1 The condition I(A:B\D) = is not needed to derive (1) and (3). We added it to make the statement weaker (and the 
result below stronger). 
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2 Example 



Consider a quadruple (A, B, C, D) q of geometric objects on the affine plane over the finite field W q : 

• First choose a random non- vertical line C defined by the equation y = cq + c\x (the coefficients Co 
and c\ are independent random elements of the field) ; 

• then pick uniformly at random a parabola D in the set of all non-degenerate parabolas y — 
do + d\x + d^x 2 (where do, ^1,^2 € F g , <^2 7^ 0) that intersect C at exactly two points; 

• and call two intersection points A and B in a random order. 

One can easily compute both sides of the inequality (*) for these A, B, C, D, and get the following 
inequality: 

l-log 2 (^)<4 K log 2 (-A_) 

This leads to a contradiction: the left-hand side is 1 — O(-) and the right-hand side is OQ) for large q. 

Informal explanations: The mutual information between the line and the parabola is approximately 
1 bit because C and D intersect at two distinct points iff the corresponding equation discriminant is a 
non-zero quadratic residue, which happens almost half of the time. Other information quantities vanish 
since the involved random geometrical objects are almost independent. 

It is in fact easy to check that they are all equal to log 2 (^zj)- F° r instance, given A there are q 
equiprobable lines C . Given A and D, there are q — 1 equiprobable lines since now the tangent to D at 
A is excluded (A ^ B). Hence I(C:D\A) — log 2 (q) — log 2 (q — 1). All other computations are similar. 

3 Historical comments and motivation 

To every quadruple of discrete random variables corresponds an entropy vector consisting of the fifteen 
entropies of all possible non-empty subsets of variables (in some fixed order) . A vector is called entropic 
if it is the entropy vector of some tuple of random variables. The set of all entropic vectors has a very 
complex structure (e.g., it is not even closed). However, the closure of the set of all entropic vectors is 
easier to study: it is a convex cone. This closure is called the cone of asymptotically entropic vectors. 

Some bounds for the cone of asymptotically entropic vectors are simple and well-known. For instance, 
these vectors satisfy Shannon inequalities, i.e., all non-negative linear combinations of the basic inequality 

H(X, Z) + H(Y, Z) - H(X, Y, Z) - H{Z) > 0. 

This inequality means that the conditional mutual information I(X:Y\Z) is non-negative. 

N. Pippenger asked in 1986 a general question [1] : does there exist any linear inequality for entropies 
other than Shannon inequalities? The first inequality of this kind was conditional [5] and appeared in 
1997, followed by a similar unconditional one [3] a year later. Quite many (actually, infinitely many) 
other inequalities have been found since then. In fact, the set of all valid information inequalities cannot 
be reduced to any finite number of "basic" inequalities: F. Matus proved that the cone of asymptotically 
entropic vectors for quadruple of random variables is not polyhedral [5]. 

Conditional equalities (1-3) express other subtle properties of entropic vectors. These inequalities 
imply conditional independence inference rules (which can not be deduced from Shannon's inequalities). 
These rules were extensively studied by several authors, see a survey by M. Studeny in 0. 

In [7] it was asked if any conditional information inequality (in particular (1)) can be obtained as 
a direct consequence of some unconditional inequality. The negative answer was given in [S]. In the 
present paper we give a simplified proof of this result, in a slightly stronger form for (1) and (3). 
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